On the Validity of the Pairs Bootstrap for Lasso Estimators

We study the validity of the pairs bootstrap for Lasso estimators in linear regression models with random covariates and heteroscedastic error terms. We show that the naive pairs bootstrap does not consistently estimate the distribution of the Lasso estimator. In particular, we identify two different sources for the failure of the bootstrap. First, in the bootstrap samples the Lasso estimator fails to correctly mimic the population moment condition satisfied by the regression parameter. Second, the bootstrap Lasso estimation criterion does not reproduce the sign of the zero coefficients with sufficient accuracy. To overcome these problems we introduce a modified pairs bootstrap procedure that consistently estimates the distribution of the Lasso estimator. Finally, we consider also the adaptive Lasso estimator. Also in this case, we show that the modified pairs bootstrap consistently estimates the distribution of the adaptive Lasso estimator. Monte Carlo simulations confirm a desirable accuracy of the modified pairs bootstrap procedure. These results show that when properly defined the pairs bootstrap may provide a valid approach for estimating the distribution of Lasso estimators. AMS (2000) Subject Classification: Primary: 62J07; Secondary: 62G09, 62E20.